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The  defense  acquisition  system  may  be  improved  through  tough,  sometimes 
controversial,  decision  making  based  on  risk.  Three  areas  of  risk  management  will  be 
examined  in  the  form  of  good  projections  of  technology  maturation,  achievable 
requirements,  and  producibility.  Technology  maturation  risk  will  examine  the 
technologies  being  pursued  by  major  defense  acquisition  programs  and  whether  the 
available  technologies  were  underdeveloped  at  Milestone  B  based  on  technology 
readiness  levels  (TRL).  The  requirements  investigation  will  examine  how  well  the 
program  was  able  to  define  requirements,  map  requirements  against  achievable  near 
term  technology,  and  keep  requirements  from  expanding  beyond  the  core  set  needed 
for  the  mission  (what  many  refer  to  as  “mission  creep”).  Producibility  risk,  defined  as 
the  ability  to  produce  end  items  within  the  projected  cost  and  schedule,  will  be 
examined  in  terms  of  overall  program  cost  and  schedule  growth  or  program 
restructurings.  Some  major  defense  acquisition  programs  as  case  examples  will  be 
examined  to  determine  influencing  factors  that  affect  program  acquisition  that  are 
beyond  the  ability  of  the  program  office  to  control.  Recommendations  are  included  for 
areas  of  further  study. 


MANAGING  UNCERTAINTY: 

RISK  MANAGEMENT  IN  ACQUISITION 


President  Obama  signed  into  law  on  May  22,  2009,  the  Weapons  Systems 
Acquisition  Reform  Act  of  2009,  and  in  his  signing  remarks  stated  “...the  Weapons 
System  Acquisition  Reforms  Act,  represents  an  important  next  step  in  this  procurement 
reform  process.  It  reforms  a  system  where  taxpayers  are  charged  too  much  for 
weapons  systems  that  too  often  arrive  late  --  a  system  that  suffers  from  spending  on 
unproven  technologies,  outdated  weapons,  and  a  general  lack  of  oversight.”1 
Acquisition  reform  has  been  ongoing  since  the  Civil  War,  to  improve  the  management  of 
the  weapons  procurement  process.2  The  acquisition  process  has  been  faulted  for  not 
adequately  addressing  immature  technologies  (“unproven”  in  the  President’s  words), 
requirements  changes  in  the  midst  of  a  program,  and  the  uncontrolled  costs  of  a 
program’s  affordability.3 

In  2008,  the  Government  Accountability  Office  (GAO)  assessed  96  programs 
with  an  average  cost  growth  of  26  percent  over  initial  acquisition  costs,  44  percent  with 
an  increase  in  per  unit  cost  greater  than  25  percent  and  an  average  schedule  delay  of 
21  months.4  Cost  overruns  and  schedule  delays  are  indications  that  programs  are 
complex  and  difficult  to  manage  as  measured  against  initial  baseline  cost  and  schedule 
estimates.  Cost  and  schedule  growth  are  symptoms  of  underlying  issues  that  add  risk 
to  program  management.  Program  risk  in  three  areas  will  be  examined  with  regard  to 
using  immature  technology,  expanding  requirements  and  the  inability  to  meet 
affordability  goals. 


This  research  paper  examines  the  risk  management  process  within  the  defense 
acquisition  process.  The  risk  management  process  will  be  defined  as  understood  by 
the  Defense  Acquisition  University,  which  is  the  training  institution  for  the  acquisition 
career  field  within  the  Department  of  Defense.5  The  risk  management  process  will  be 
examined  for  the  ability  to  address  technology  risk,  requirements  change,  and  the  cost 
or  affordability  of  programs. 

Risk  Management 

Risk  is  a  measure  of  future  uncertainty  in  the  ability  or  inability  to  achieve 
program  objectives  within  costs,  schedule,  and  technical  performance  parameters.6 
Risk  has  two  components,  a  present  probability  that  a  future  risk  will  occur  and  a 
consequence  or  effect  if  that  future  risk  occurs.7  So,  risk  management  is  about 
managing  future  events,  and  is  not  about  managing  current  issues  or  present  problem 
solving.8  Risk  management  is  about  prioritization  and  concentrating  program  resources 
on  the  high  impact,  high  probability  risk.9  As  such,  it  proactively  informs  the  Program 
Manager  on  where  to  spend  valuable  resources,  money,  time,  people  or  technology, 
rather  than  reacting  to  problems  as  they  arise.  Risk  management  is  a  continuous 
process  of  managing  perceived  future  risk  to  the  program  by  a  multi-function  team. 

Risk  management  consists  of  four  major  steps  that  examine  risk:  risk  planning, 
risk  assessment,  risk  handling  and  risk  tracking  (see  figure  1 ).  Risk  planning  is 
concerned  with  implementing  a  strategy  that  develops  a  team-based,  collaborative 
process,  a  documented  and  agreed  upon  method  to  identify,  categorize,  track  and 
revisit  risks  throughout  the  life  cycle  of  a  program.  A  plan  should  include  a  method  to 
address  level  of  severity,  probability  of  occurrence,  and  responsibilities  of  team 
members,  and  frequency  of  meeting.  The  plan  should  also  document  how  to  initiate 
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the  tracking  and  identification  of  new  risks.  Because  risk  is  not  limited  to  the  technical 
management  of  a  program,  team  members  should  be  from  every  portion  of  the 
acquisition  community  and  program  management  office:  management,  procurement, 
resource  management,  technical  or  engineering,  security,  legal,  acquisition,  test, 
requirements  and  the  industry  partners  where  appropriate.11  Areas  that  may  be 
addressed  in  a  risk  management  strategy  include  budget,  schedule,  performance, 
contracting,  political  will,  strategic  direction,  internal  meeting  and  communications 
processes,  technology,  requirements,  manufacturing  technology,  software,  integration 
complexity,  material,  work  breakdown  structure,  decision  processes,  documentation, 
and  quality  control  to  mention  the  most  common.12  Not  all  of  these  are  required  or 
necessary;  they  are  project  dependent. 

Risk  Assessment  involves  three  steps:  risk  identification,  risk  analysis  and  risk 
prioritization.  Risk  identification  is  based  on  program  requirements  from  various 
sources  including  the  user’s  requirements,  contracting,  testing,  performance  work 
statements,  design  drawings,  integration,  technology  development,  manufacturing 
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technology  as  well  as  the  objectives  for  cost,  schedule  and  performance  and  other 
source  documentation.13  The  program  objectives  in  technical,  operational  and  business 
processes  are  assessed  for  sources  of  risk,  both  internal  and  external.14  Internal  risks 
are  within  the  scope  of  the  program  to  manage,  such  as  ensuring  interoperability. 
External  risks  are  those  beyond  the  internal  control  of  the  program  management  office, 
such  as  a  political  shift  that  may  impact  funding.  The  intent  of  any  risk  identification  and 
analysis  is  to  identify  the  root  cause  of  the  risk,  and  associate  it  with  the  part  of  the 
program’s  work  breakdown  structure  where  the  risk  will  be  managed.15  An  analysis 
provides  a  concise  statement  of  what  the  risk  is  and  what  its  future  impact  is  for  that 
element  of  the  work  breakdown  structure  that  has  the  risk  assigned.16  These  risks  are 
managed  using  a  risk  register  which  is  reviewed  periodically,  and  can  be  computer- 
based  programs.17  Risks  occur  throughout  the  program,  from  early  on  with  technology 
risk  to  later  on  in  production  as  manufacturing  technology  is  more  relevant.18  The 
process  for  maintaining  a  risk  register  is  a  part  of  the  responsibilities  of  the  risk 
management  team.  The  register  contains  the  risk,  consequence,  and  action  needed  to 
address  the  risk.19 

The  last  step  in  the  assessment  process  is  risk  prioritization.  To  manage  the 
risk,  a  prioritization  method  is  used  to  rank  the  probability  of  a  risk  occurring  and  a  level 
of  potential  impact  or  consequence.20  For  instance,  if  the  risk  is  very  low  and  the 
consequence  or  impact  to  the  program  is  very  high,  resources  may  be  allocated  to 
measure  and  monitor  the  risk  and  not  allocated  immediately  to  implement  action  to 
mitigate  the  risk.  Various  scales  and  ranking  methods  are  used.  The  scales  are 
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associated  with  thresholds  established  by  the  risk  management  team  for  the  level  of 
consequence  that  is  considered  acceptable.21  This  process  allows  a  risk 
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Figure  2.  Example  Risk  Matrix  with  Probability  and  Consequence  Definitions.22 


matrix  to  be  displayed  that  cross  references  high  likelihood  and  high  consequence  risk 
to  clearly  identify  risks  that  may  need  management  plans  to  mitigate.  See  Figure  2  for 
an  example  matrix,  associated  definitions  for  probability  and  consequence,  and  the 
endnotes  for  a  more  detailed  explanation.  Prioritization  assists  the  risk  management 
team  to  assess  the  projected  risk  and  allocate  its  resources  on  the  more  probable  and 
realistic  risks  that  may  have  an  impact  on  the  program.23 

Once  the  risks  have  been  identified  and  prioritized,  the  risk  management  team 
and  the  program  management  office  are  able  to  plan  to  handle  the  risks.  Risk  handling 
involves  developing  mitigation  strategies  for  reducing  the  occurrence  or  severity  of  the 
identified  risks.  The  possible  strategies  are  avoidance,  transference,  control  and 
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assumption,  either  used  by  themselves  or  in  combinations.  The  avoidance  strategy 
seeks  to  choose  alternative  approaches  to  the  cost,  schedule  or  performance  risk  in 
order  to  eliminate  the  risk  from  occurrence,  for  instance,  choosing  a  less  risky 
technology  concept.  Transference  is  about  reallocating  risk  within  the  system,  and 
relies  on  system  design  techniques,  such  as  modularity,  to  transfer  risk  to  a  single 
module  or  function  in  order  to  concentrate  resources  on  that  function  or  module.  Risk 
control  is  about  reducing  or  mitigating  risk  to  reduce  the  probability  of  the  risk  occurring 
or  lessen  the  impact  on  the  program.  An  example  of  risk  control  is  using  multiple 
development  efforts  to  develop  several  possible  designs  so  that  if  one  design  proves 
intractable  to  develop,  other  designs  are  available  to  achieve  the  same  desired 
performance.  Lastly,  risk  assumption  is  acceptance  of  the  risk  and  is  usually  reserved 
for  consequences  and  probabilities  that  are  relatively  low,  recognizing  that  program 
resources  are  better  used  on  the  higher  risk  categories.  The  plan  uses  these  strategies, 
and  then  uses  risk  tracking  through  the  use  of  appropriate  metrics  to  monitor  risk 
occurrence. 24 

Risk  tracking  uses  the  results  of  metrics  developed  for  use  that  will  ideally  be 
constructed  to  indicate  early  warning  of  risk,  based  on  cost,  schedule  and  performance 
or  other  risk  category.  Some  measurement  techniques  are  test,  earned  value 
management,  schedule  performance,  program  metrics  and  technical  performance 
measurements  among  others.  Risk  tracking  may  also  discover  additional  risks,  which 
then  may  be  addressed  with  a  robust  risk  management  process,  to  establish  a  risk 
level.25 
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Risk  management  is  another  tool  for  the  program  manager,  to  inform  and  be 
tailored  based  on  the  needs  of  the  program.  The  process  provides  a  structured  way  to 
examine  risk  and  provide  information  on  a  continuous  basis  to  make  informed 
decisions.26  Risk  management  does  not  replace  the  attention  to  detail  and  structured 
decision  process  required  in  well  managed  acquisition  programs.27 

Analyses  of  defense  and  public  sector  infrastructure  programs  demonstrate  that 
all  programs  have  risk  and  that  risk  is  manageable.28  Cost  and  Schedule  variance  are 
common  program  management  metrics  used  to  monitor  program  progress.29  The 
manifestations  of  risk  as  schedule  delays,  cost  increases  or  performance  shortfalls  are 
common  to  all  large  projects;  defense  and  public  sector.30  In  the  major  studies  of 
defense  related  programs,  risk  is  categorized  into  three  areas  that  the  program 
managers  may  be  able  to  influence:  technology  selection  and  use,  realistic 
requirements  generation  and  producibility  or  cost  effectiveness.31 
Technology  Risk 

Technology  risk  is  risk  associated  with  choosing  technologies,  integration  of 
technologies  or  software  development,  or  manufacturing  technologies.  Within  the  DOD, 
the  technology  risk  for  major  parts  of  programs  are  assessed  by  ranking  the  technology 
against  a  preset  scale  from  one  to  nine  in  defined  technology  readiness  levels  (TRL), 
with  nine  being  the  most  mature  and  proven  technology  in  real  mission  environments 
and  one  being  basic  research.32  To  be  assessed  as  a  mature  technology  ready  for  the 
engineering  and  manufacturing  development  (EMD)  phase,  a  TRL  level  of  six  or  above 
is  desired,  prior  to  entering  into  system  development.33 

To  illustrate  technology  risk,  consider  five  programs,  consisting  of  the  Army 
Tactical  Missile  System  (ATACMS),  a  deep  strike  missile,  Javelin,  an  anti-tank  missile, 
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the  Future  Combat  System  (FCS),  a  suite  of  systems,  the  Joint  Direct  Attack  Munition 
(JDAM),  a  guidance  package  for  unguided  bombs,  and  the  F-22  Raptor,  an  advanced 
tactical  fighter.  The  technology  risk  for  these  programs  will  be  examined  in  terms  of 
programmatic  symptoms  of  risk  in  schedule  and  cost  impacts  and  program  restructures. 

Table  1  provides  a  comparison  of  selected  programs.  ATACMS  and  JDAM  were 
two  programs  that  entered  the  EMD  phase  with  a  more  significant  portion  of 
technologies  at  level  six  and  above.  ATACMS  and  JDAM,  albeit,  smaller  in  scope  than 


Program 

Aspects 

ATACMS 

Javelin 

FCS 

JDAM 

F-22 

Evolutionary 

Single-Step 

Evolutionary 

Single  Step 

Evolutionary 

Ultimate 

Capability 

Deep  Attack 

Fire  &  Forget 

Networked  Force 

Precision  Guidance 

Tactical  Fighter 

Subsystem 

Critical 

TRL  >  6 

Critical 

TRL  >6 

Critical 

TRL  >6 

Critical 

TRL  >6 

Critical 

TRL  >6 

(at  Milestone 
B) 

6 

6 

7 

0 

31 

1 

6 

6 

unknown 

unknown 

(4  at  9) 

(5<6) 

Cost  of 
Development 

-S700M 

-S700M 

-S22B 

-S380M 

-$11B 

Months 

Months 

Months 

Months 

Months 

Tech 

Development 

Phase 

0 

27 

16 

0 

54 

Advanced 
Development 
Phase  — 

Planned 

48 

36 

32 

46 

41 

Advanced 
Development 
Phase  — 

Actual 

51 

54 

56 

40 

59 

Total  Time  in 
Development 

51 

81 

113 

Schedule 

Growth 

6% 

50% 

57% 

0 

44% 

Cost  Growth 

0% 

150% 

76% 

-25% 

85% 

EMD 

Program 

Restructures 

unknown 

unknown 

5 

1 

5 

Table  1 .  Program  Technology  Readiness  and  Cost  and  Schedule  Growth.34 
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FCS  and  the  F-22,  experienced  significantly  smaller  or  no  schedule  and  cost  overruns. 
ATACMS  and  JDAM  used  technologies  that  were  mostly  well  understood  prior  to 
entering  EMD.  ATACMS  borrowed  the  M74  bomblet,  solid  rocket  motor,  fin  surface 
flight  control  and  inertial  guidance  system  technologies  from  the  family  of  Multiple 
Launch  Rocket  System  (MLRS)  rocket  programs,  which  have  been  in  development 
production  since  1983. 35  All  of  these  are  proven  TRL  level  9  technologies  that  were 
incorporated  into  the  early  stages  of  the  ATACMS  program.36  Likewise,  JDAM  used 
more  mature  technology  at  the  outset,  over  85  percent  of  the  JDAM  components  were 
from  commercial  sources  already  producing  the  technology.37 

In  contrast,  Javelin,  FCS  and  the  F-22  had  several  immature  technologies  on 
entry  into  the  equivalent  EMD  phase.  Javelin  entered  EMD  with  no  critical  technologies 
at  TRL  level  6. 38  The  Javelin  program  was  using  a  single-step  program  within  EMD  to 
achieve  a  revolution  in  seeker  technology  with  imaging  infrared  and  software-driven 
automatic  target  tracking  and  engagement,  and  a  missile  with  soft-launch  capability, 
neither  of  which  was  available  at  the  beginning  of  the  program.39  As  a  consequence, 
the  Javelin  program’s  development  schedule  was  lengthened  by  2  years  and  cost 
growth  was  150  percent  higher  than  the  initial  baseline.40  FCS  was  into  the  program’s 
fifth  year,  after  four  years  of  technology  development  and  a  full  year  into  EMD,  before 
technology  risk  mitigation  plans  were  even  in  place  for  the  critical  technologies  needed 
to  produce  FCS.41  Even  at  milestone  B  in  2003,  FCS  had  31  identified  critical 
technologies  where  30  were  below  level  6  or  system  or  sub-system  demonstration  in  a 
relevant  environment.42  The  number  of  critical  technologies  increased  to  49  in  2006 
with  18  reaching  TRL  level  6. 43  As  late  as  2008,  the  FCS  network  architecture 
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technology  was  assessed  as  immature.44  The  F-22  program  technology  risks  involved 
avionics  integration,  a  new  supercruise  engine  with  thrust  vectoring  and  a  low  radar 
cross  section  airframe  with  composites.  The  airframe  and  avionics  experienced  42  and 
25  percent  cost  growth  from  1996  to  2002,  respectively.45  The  program  also 
experienced  schedule  delays  of  up  to  a  year  for  delivery  of  airframes  due  to  airframe 
manufacturing  difficulties.46  The  technology  associated  with  these  programs  was 
immature  technology  at  Milestone  B  and  has  a  correlation  with  program  cost  and 
schedule  overruns. 

Although  technology  alone  is  not  responsible  for  cost  and  schedule  overruns, 
evidence  of  immature  technology  contributing  to  program  risk  exists.  When  this  is 
coupled  with  management  decisions  to  allow  entry  into  EMD  with  immature  technology, 
or  is  not  accounted  for  in  risk  management  and  planning,  program  costs  and  schedules 
begin  to  increase.  Management  decisions  that  impacted  programs  negatively  occurred 
in  both  the  FCS  and  F-22  programs.  In  the  FCS  program,  risk  management  was  not 
complete  until  a  full  year  into  the  EMD  phase  of  the  program.47  The  delayed  effort  at 
risk  management  planning  coupled  with  the  32  of  33  immature  technologies  contributed 
to  cost  and  schedule  increases  that  ultimately  led  to  five  program  restructurings.  FCS 
entered  EMD  with  immature  technologies  with  the  agreement  of  the  decision  makers. 
The  F-22  program,  in  comparison  to  the  F/A-18  E/F  program,  had  only  a  small 
management  reserve,  and  was  operating  on  the  over-optimistic  planning  that 
technology  was  not  a  risk,  which  proved  inadequate  for  the  airframe  and  the  avionics 
integration,  both  of  which  caused  significant  delays  in  the  program.48  In  order  to  enter 
into  development,  technology  assessments  for  maturity  should  be  included  as  part  of 


10 


the  review  needed  for  Milestone  B  decisions.  Early  in  the  program,  prior  to  milestone  B, 
technology  maturity  should  be  an  element  of  the  risk  management  plan  and  added  to 
the  risk  management  process.  Although  immature  technology  is  a  risk  that  contributes 
to  program  cost  and  schedule  overruns,  other  areas  of  risk  impact  programs,  such  as 
changing  requirements  in  the  midst  of  EMD. 

Requirements  Risk 

Requirements  risk  is  often  due  to  requirements  instability.  Requirements 
instability  is  represented  as  poorly  defined  and  changing  requirements  during  the 
development  of  the  system.  In  DOD  and  intelligence  community  (1C)  programs, 
requirements  instability  has  contributed  directly  to  cost  and  schedule  growth.49  In  the 
case  of  the  Space-Based  Infrared  System  High  (SBIRS-High),  new  requirements  were 
being  added  well  into  development,  and  were  poorly  defined  at  the  start  of 
development.50  A  billion  dollar  cost  growth  to  the  program  was  attributed  to 
requirements  growth  and  definition.51  In  the  FCS  program,  three  years  into  product 
development  (2006),  system  level  requirements  were  still  in  flux.52  Requirements 
should  be  well  defined  at  Milestone  B.  In  the  Advanced  Extremely  High  Frequency 
(AEHF)  communications  satellite  program,  cost  growth  of  $0.72  billion  was  attributed  to 
requirements  that  were  added  to  the  program  after  May  of  2000. 53  In  the  Joint 
Surveillance  and  Target  Attack  Radar  System  (JSTARS)  Common  Ground  Station 
(CGS),  not  only  were  requirements  added,  two  versions  of  the  station,  a  light  and 
medium,  were  added  to  the  program  after  development  was  underway.54  One  version 
was  terminated  for  affordability  reasons  after  a  few  years  in  development.55  In  a  study 
of  13  Army  programs,  those  that  experienced  requirements  changes  in  mid 
development  experienced  50  percent  less  favorable  performance  outcomes  than  those 
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programs  that  did  not  experience  requirements  changes. 56  Favorable  performance 
outcomes  were  defined  as  meeting  technical  requirements  and  cost  and  schedule 
goals.57 

Requirements  instability  impacts  program  cost  and  sometimes  schedule  growth. 
However,  holding  requirements  firm,  resisting  increases  in  requirements,  and  holding 
life  cycle  times  to  short  time  periods,  improve  the  chances  of  program  performance  on 
cost  and  schedule.58  Those  programs  that  met  cost  and  schedule  goals  were  able  to 
achieve  two  outcomes:  short  development  times  and  clearly  defined  requirements  at 
program  development  start.59  Requirements  as  a  source  of  instability  should  be 
considered  within  the  context  of  risk  planning  and  an  effort  made  to  control  increases  to 
existing  well  defined  requirements. 

The  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  is  the  DOD 
requirements  approval  and  definition  process  that  approves  major  defense  acquisition 
program  requirements.60  The  JCIDS  process  has  been  characterized  as  slow  to 
achieve  approval  as  well  as  being  service  parochial.61  Most  programs  are  validated  by 
the  Joint  Requirements  Oversight  Council  (JROC).62  The  requirements  approval 
process  is  not  linked  with  the  ability  of  the  acquisition  community  to  provide  feasible 
technology  within  funding  constraints.63  DOD  is  implementing  a  Milestone  Decision 
Authority  (MDA)  review  at  program  initiation  to  examine  technology  feasibility  and 
affordability,  as  outlined  in  the  2008  DOD  Instruction  5000. 02. 64  Efforts  at  linking  the 
funding,  requirements  and  acquisition  processes  for  better  informed  decision  making 
have  potential  to  improve  the  complex  environment  that  procures  materiel  for  the  forces. 
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Producibility  Risk 

Producibility  is  defined  as  meeting  cost  and  schedule  goals.  Risk  increases 
when  costs  or  schedules  increase  buying  less  weapon  systems  for  the  funding  spent. 
Several  GAO  reports  have  demonstrated  program  cost  growth  is  due  to  requirements 
instability  or  technology  immaturity  or  both  and  that  cost  growth  is  a  symptom  of  these 
underlying  causes.65  In  general,  cost  growth  in  a  program  is  identified  by  schedule 
delays,  increased  costs  per  unit  production,  or  less  production  for  the  funding.66  The 
FCS  and  F-22  programs  demonstrated  cost  growth  from  immature  technologies. 
Requirements  increases  affected  the  cost  of  several  satellite  programs.67  The  cost 
increase  has  in  a  number  of  programs  resulted  in  fewer  production  end  items  that 
increase  the  average  per  unit  procurement  cost  (AUPC).68  Other  factors  that  may 
contribute  to  growth  in  AUPC,  are  unrealistic  initial  costs  driven  by  the  “low-bid” 
syndrome.  The  low  bid  syndrome  is  the  desire  by  a  contractor  to  procure  the  contract 
by  producing  an  unrealistic  baseline  cost  for  the  development  of  the  program.69  The 
consolidation  of  the  defense  industry  may  be  exacerbating  the  bidding  process.  The  bid 
process  may  not  be  an  accurate  reflection  of  the  true  cost  of  the  program.  Another 
possible  cause  of  increased  AUPC  is  the  financial  instability.  If  the  program  was 
funded,  and  that  funding  was  reduced  from  external  causes,  the  AUPC  will  increase 
when  less  funding  is  available  and  quantities  are  reduced.70  Other  effects  from  funding 
instability  include  personnel  turnover  and  testing  schedule  changes  that  often  result  in 
increasing  costs  or  schedules  or  both.71  Affordability  is  impacted  by  funding  stability, 
outside  intervention  into  a  programs  funding,  the  low-bid  process. 

Cost  and  schedule  increases  to  program  affordability  are  not  themselves  root 
causes  of  affordability  risk.  The  identification  with  root  causes  for  affordability  risks  are 


13 


essential  in  being  able  to  determine  the  increasing  lack  of  affordability  in  DOD  weapons 
acquisition  programs.  The  identification  of  the  root  cause  of  the  funding  instability  aides 
in  determination  of  risk  mitigation  actions.  For  example,  like  the  FCS  and  F-22,  the 
Stryker  and  Javelin  programs  experienced  affordability  issues.  In  the  Stryker  program 
technical  armor  and  weight  issues  with  the  Mobile  Gun  System  (MGS)  were  the  root 
cause  of  cost  increase.72  In  the  Javelin  program,  the  seeker  technology  maturity  was 
the  root  causes  of  the  program  affordability  issues. 73  Both  programs  had  cost  and 
schedule  increases.  The  cost  and  schedule  increases  are  metrics  that  allow  a  program 
manager  to  track  affordability  risk  of  weapons  programs  and  should  be  monitored  as 
indicators.  They  can  be  risk  mitigated,  by  allocating  management  funding  reserves  and 
time  reserves  in  the  early  planning  of  the  schedule. 

The  Planning  Programming  Budgeting  and  Execution  (PPBE)  system  also  has 
impacts.  In  the  case  of  the  F-22  the  AUPC  was  pushed  upward  by  decisions  to 
decrease  from  750  to  648  aircraft  in  1990,  and  then  to  442  in  1994,  and  then  to  339  in 
1997  when  congress  imposed  a  cost  limit  to  production  for  affordability  reasons  as  costs 
for  the  program  continued  to  rise.74  The  final  numbers  were  based  on  existing  budget 
and  capped  the  program  at  187,  with  the  recommendation  of  Defense  Secretary 
Gates.75  Both  program  costs  and  decisions  from  outside  the  program  office  increased 
the  AUPC. 

Risk  Management  and  Managerial  Influence 

In  a  few  cases,  programs  have  managed  to  achieve  performance  cost  and 
schedule  outcomes.  In  those  cases  that  achieved  favorable  outcomes,  JDAM, 

ATACMS  and  MRAP,  risk  mitigation  strategies  that  contributed  to  favorable  results  will 
be  examined. 
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JDAM  employed  a  number  of  techniques  at  risk  mitigation.  One  was  to  ensure 
that  the  program  held  a  rigid  mission  performance  and  requirements  process  consisting 
of  seven  fixed  and  untradeable  Key  Performance  Parameters  (KPPs)  to  avoid  increases 
or  changes  in  requirements.76  This  approach  requires  two  methods,  a  well  defined 
initial  set  of  requirements  as  a  baseline,  and  no  new  requirements  after  the  Preliminary 
Design  Review  (PDR).77  Another  technique  employed  by  the  JDAM  program  was  to 
rigorously  avoid  any  effort  at  over-design  from  the  contractor  development,  using  85 
percent  commercial  and  government  off-the-shelf  technologies.78  The  use  of  these 
technologies  also  contributed  to  the  ability  of  the  program  to  use  mostly  mature 
technologies.79  Affordability  risk  was  controlled  through  a  rigid  cost  control  program  that 
added  Average  Unit  Production  Price  (AUPP)  as  a  KPP.80  The  use  of  commercial  or 
GOTS  sources  provided  the  government  with  cost  benefit  trades  and  alternatives  to 
meet  the  AUPP  KPP.81  The  affordability  of  JDAM  using  average  unit  price  may  be 
because  JDAM  is  an  expendable  package  with  low-to-no  maintenance  needs. 

JDAM  implemented  two  processes  under  acquisition  reform  that  transferred  risk 
to  the  contractor:  configuration  control  and  a  warranty.82  The  contractor  was 
responsible  to  maintain  designs,  and  to  provide  a  warranty  on  performance  of  20  years 
shelf  live  and  5  years  service  life,  accepting  the  risk  of  maintaining  quality  to  comply 
with  the  warranty.83  The  use  of  Integrated  Process  Teams  (IPTs)  in  close  partnership 
with  industry  to  provide  immediate  feedback  on  performance  from  both  test  programs 
and  design  performance  enabled  changes  on  the  contractors’  designs  to  reduce  cost 
and  increase  performance.84  The  IPT  process  paid  dividends  when  the  program 
underwent  the  only  restructuring  as  a  result  of  flight  instability  on  the  Mk-83  and  BLU- 
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109  JDAM  kits  discovered  in  testing,  delaying  kits  for  about  one  year.85  The  IPTs 
resolved  the  flight  instability.  As  will  be  shown  later  this  same  process  provided  benefits 
on  MRAP.  The  Joint  Air-to-Surface  Standoff  Missile  (JASSM)  is  another  missile 
program  that  used  similar  techniques,  like  using  proven  technology  in  the  engine  and 
imaging  sensor,  solidifying  requirements,  and  making  cost  a  KPP,  all  of  which  achieved 
similar  results.86 

ATACMS  used  technologies  that  were  very  mature,  TRL  levels  of  6  or  higher, 
with  many  at  TRL  level  9,  having  been  borrowed  from  the  MLRS  program.  ATACMS 
uses  a  modular  independent  architecture  compatible  with  the  M270A1  and  High  Mobility 
Artillery  Rocket  System  (HIMARS)  fire  control  and  launchers,  building  on  the  MLRS 
program.87  The  use  of  mature  technologies,  a  well  defined  and  limited  set  of 
requirements  and  the  short  duration  of  the  program  aided  the  ATACMS  program  office 
in  achieving  production  on  schedule  and  within  budget.88  ATACMS  experienced  no  late 
engineering  changes,  achieved  system  unit  costs,  met  the  program  technical 
requirements,  and  experienced  no  significant  issues  during  deployment.89 

The  Mine  Resistant  Ambush  Protected  (MRAP)  vehicle  program  implemented  a 
risk  management  strategy  to  mitigate  risk  to  the  program.90  In  order  to  achieve 
production  quantities,  multiple  vendors  with  multiple  designs  were  needed.91  The  test 
and  evaluation  program  was  designed  as  a  risk  reduction  and  mitigation  part  of  the 
program.92  The  test  and  evaluation  was  designed  to  test  the  most  critical  capabilities 
first,  followed  by  others  later.93  As  part  of  the  process,  contractor-1  PTs  were  conducted 
by  the  program  office  to  identify  any  changes  that  could  be  incorporated  into  the 
designs  for  the  next  block  of  production,  so  a  test-fix-test  program  was  used  to  provide 
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risk  reduction. 94  Also,  MRAP  identified  two  material  flows  early  in  the  development  that 
could  impede  production  -  domestic  armor  supply  and  tires  rated  to  fifteen  tons.95  Law 
requires  that  domestic  armor  be  procured  for  military  vehicles  from  either  domestic 
suppliers  or  Canada.96  The  program  obtained  a  waiver  to  procure  armor  from  overseas 
vendors  in  order  to  achieve  production  quantities.97  Tires  were  another  risk  identified. 
The  program  approached  a  second  manufacturer  to  produce  additional  tires,  and 
approached  the  Defense  Logistics  Agency  (DLA)  to  produce  more  tire  molds.98  Both 
were  needed  to  produce  tires  for  production  and  as  spares  as  MRAP  vehicles  were 
fielded.99 

MRAP  had  other  material  requirements  and  approached  the  DOD  to  receive  a 
DX  rating  to  compete  with  other  major  programs  for  materials  and  spares.100  The 
program  anticipated  integration  as  risk.  The  program  approached  Space  and  Naval 
Warfare  Systems  (SPAWAR)  Charleston  to  assemble  Government  Furnished 
Equipment  (GFE)  into  MRAP  vehicles.  SPAWAR  effectively  became  a  central  shipping 
point  for  installation  of  all  GFE  and  attained  integration  rates  of  50  vehicles  a  day.101 
With  Charleston  Air  Force  Base,  an  airlift  capable  airfield,  and  Charleston  Naval  Base,  a 
naval  port  that  allowed  MRAPs  to  be  quickly  shipped  as  production  approached  a 
thousand  vehicles  per  month,  within  six  miles  of  SPAWAR,  Charleston  was  an  ideal 
central  integration  site.102 

Since  MRAP  was  DOD’s  first  priority  program,  many  of  the  oversight  and 
decision  processes  were  assisted  by  the  program’s  priority.  The  amount  of  funding  has 
propelled  MRAP  to  DOD’s  third  largest  program,  with  decisions  yet  to  be  made  on  the 
disposition  of  MRAPs  in  the  post  OIF  materiel  force  structure,  the  Marine  Corps  has 
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already  stated  MRAPS  are  too  heavy  for  their  expeditionary  warfare.103  Also,  other 
tactical  wheeled  vehicles,  such  as  the  Joint  Light  Tactical  Vehicle  (JLTV),  have  been 
extended  in  early  technology  development  as  MRAPs  are  fielded  to  meet  the  current 
need.104 

The  program  manager  can  achieve  cost  and  schedule  goals  as  demonstrated  by 
the  JDAM,  ATACMS  and  MRAP  case  studies.  However,  significant  factors  influence 
the  ability  of  DOD  and  the  program  manager  to  achieve  those  goals.  Funding  stability 
so  that  the  program  is  not  required  to  be  a  “program  promoter”  to  secure  funding  for  the 
program  in  the  annual  funding  cycle  of  DOD  would  help  promote  stability. 105  The 
incentive  structure  is  currently  based  on  schedule,  cost  and  performance  success,  not 
necessarily  valid  projections  of  risk  or  difficulty  or  providing  a  valid  program  baseline.106 
The  tenure  and  attraction  of  talent  are  also  dependent  on  stable  funding  and  the 
personnel  and  management  chains  on  projects  should  remain  in  place  from  the 
inception  of  the  project  through  the  development  phase,  as  industry  practices.107  The 
DOD  acquisition  management  chain  approves  through  the  requirements  process  more 
programs  than  DOD  is  able  to  afford.  The  DOD  acquisition  management  chain  should 
examine  portfolio  management  as  a  means  of  providing  funding  decisions  on  the 
services’  acquisition  programs,  using  the  QDR  process  to  determine  programs  that  will 
receive  funding  over  the  long  term  to  provide  funding  stability.108 

Accountability  is  also  important  in  both  public  sector  and  defense  programs,  to 
insure  public  sector  funds  are  achieving  the  performance  and  requirements  originally 
designed  for  the  programs.  Oversight  is  a  government  responsibility  and  should  be  a 
function  of  the  program  office;  the  program  office  should  be  the  program  monitor,  not 
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the  program  promoter.109  Several  reforms  to  achieve  more  accountability  are  to  ensure 
the  government  is  providing  oversight  on  the  program,  not  promoting  the  program,  and 
to  adopt  industry  practices  for  program  accountability  by  assuring  the  program 
management  is  in  place  for  the  duration  of  the  development  of  the  program.110 
Conclusions 

Risk  management  and  early  decision  making  need  to  go  hand  in  hand.  Most  of 
the  crucial  decisions  are  made  in  the  early  phase  of  development,  prior  to  milestone 
B.111  Good  decision  processes  must  accompany  good  risk  mitigation  strategies  and 
acquisition  strategies  require  tailoring  to  the  complexity  and  needs  of  the  program; 
however,  some  commonalities  exist.  Technology  risk  in  the  form  of  technology 
immaturity  and  requirements  risk  in  form  of  poorly  defined  or  changing  requirements 
have  occurred  in  a  number  of  programs  generating  cost  and  schedule  overruns. 
Although  producibility  risk  occurs  and  may  be  risk  mitigated,  more  often,  cost  and 
schedule  variances  are  evidence  of  underlying  issues  such  as  technology  immaturity  or 
requirements  instability. 

The  three  areas  examined,  technology,  requirements  and  producibility  are 
important  risks  to  be  examined  in  a  program.  The  program  manager  and  decision 
makers’  ability  to  manage  and  control  requirements  is  possible  with  courageous 
decisions  to  not  allow  programs  to  enter  EMD  with  either  poorly  defined  requirements  or 
those  that  cannot  be  easily  translated  into  technical  and  system  solutions,  and  to  hold 
requirements  firm  over  definitive  time  periods.  In  cases  where  system  development 
may  take  significantly  longer  than  four  or  five  years,  evolutionary  acquisition  with 
additional  milestones  may  help  control  requirements  instability.  Albeit,  not  solely  in  the 
domain  of  the  program  manager,  requirements  growth  should  be  discouraged  once 
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EMD  begins.  Likewise,  with  technology  immaturity,  risk  may  be  controlled  by 
measuring  technology  maturity  and  only  allowing  programs  to  enter  EMD  with  TRL 
levels  of  6  or  higher  for  significant  portions  of  the  critical  technologies. 

Cost  and  schedule  overruns  occur  on  other  than  defense  related  projects. 
Railroad,  highway  and  other  large  infrastructure  projects  such  as  canals  and  opera 
houses  are  typically  over  cost  by  significant  margins,  from  anywhere  on  cost  to  as  much 
as  200  percent  overruns.112  Large  software  projects  experience  similar  overruns.113  In 
many  of  these  projects,  as  demonstrated  here  for  defense  projects,  optimistic 
projections  of  value  and  use  are  at  the  root  cause  of  the  inflated  viability  of  the  projects. 
As  in  public  sector  large  projects,  complex  defense  projects  are  overly  optimistic  in  the 
expected  costs  and  ability  of  technology  to  provide  leaps  in  performance  in  the  early 
stages  of  development.114  For  transportation  infrastructure  projects,  optimism  results  in 
a  belief  in  unrealistic  forecasts  of  usage,  undervaluing  environmental  impacts,  and  over¬ 
inflating  the  regional  economic  benefits  for  transportation  and  large  infrastructure 
projects.115 

The  development  of  a  risk  management  plan  based  on  the  most  likely  scenarios 
as  opposed  to  the-everything-goes-according-to-plan  approach  is  recommended  in 
order  to  provide  more  realism  in  the  cost  estimation  of  the  project.116  Managing  risks 
can  help  programs  achieve  goals  on  cost,  schedule  and  performance.  Ultimately,  the 
government  benefits  from  the  proper  use  of  risk  management  to  prepare  for  and 
mitigate  risk  to  programs.  By  achieving  these  goals,  the  DOD  is  better  prepared  to 
estimate  total  program  cost.  The  defense  department  can  achieve  better  project 
performance  with  risk  management.  By  including  risk  analysis,  the  feasibility  of  a 
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project  based  on  realistic  costs  over  the  life  of  the  project  will  provide  a  better  estimation 
of  the  total  cost  of  the  program  or  project  over  the  program’s  life  cycle.  The  Norwegian 
defense  agency  is  adopting  just  such  an  approach.117 

The  DOD  acquisition  system  is  not  meant  to  be,  nor  should  it  be,  a  “one  size  fits 
all”  approach.  Software  development  is  different  from  large  infrastructure  building 
development  is  different  from  technologically  mature  weapons  systems  is  different  from 
technologically  complex  systems.  Tailoring  the  program  to  specific  needs  is  the  type 
flexibility  that  DOD  requires  to  manage  programs.  Some  large  complex  programs  will 
need  significant  risk  management  programs,  other  less  technologically  complex 
programs  may  not.  Complex  software  development  efforts,  and  other  non-defense 
programs  also  experience  cost  and  schedule  overruns.  Risk  management  may  mitigate 
some  of  the  risks  in  a  program.  However,  risk  management  is  not  a  single  source 
solution  for  the  complex  management  of  uncertainty  in  a  program.  Risk  management 
when  coupled  with  other  techniques  such  as  decision  analysis  may  help  a  program 
manager  in  early  identification  of  potential  sources  of  cost  and  schedule  growth  as  parts 
of  a  robust,  continuous  management  approach. 

Recommendations  for  Further  Investigation 

In  some  of  the  case  studies  examined,  the  decision  processes  that  have  allowed 
programs  with  poorly  defined  or  changing  requirements  or  immature  technologies  to 
enter  into  EMD  is  an  area  of  investigation  that  would  be  beneficial  to  DOD.  The 
implication  of  the  FCS  program  is  that  the  program  was  entered  into  EMD  with  poorly 
defined  requirements  and  immature  technologies,  and  both  were  known  prior  to  entry 
into  EMD.  The  decision  processes  that  lead  to  these  conditions  should  be  examined. 
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Some  research  infers  that  the  Planning,  Programming,  Budgeting  and  Execution 
System  (PPBES)  and  program  advocacy  by  program  acquisition  personnel  lead  to  poor 
decision  analysis  with  Milestone  Decision  Authorities  (MDA).118  Others  suggest  that  risk 
adverse  managers  are  driving  poor  decisions  and  that  the  system  promotes  limited 
information  flow  to  decision  makers.119  Budget  driving  decisions  and  success  oriented 
programs  are  areas  in  need  of  further  investigation. 

Some  analyses  have  correlated  funding  instability  with  personnel  turnover.120  An 
area  of  further  study  would  be  the  correlation  of  turnover  and  tenure  with  program 
success.  If  more  responsibility  is  delegated  to  program  managers  to  control  technology 
and  requirements,  some  thought  about  how  long  the  program  managers  should  be  with 
the  program,  perhaps  from  advanced  development  through  EMD  should  be  examined 
with  program  success.  That  same  tenure  investigation  might  be  applied  to  the 
acquisition  decision  makers  in  the  decision  chain. 

The  certification  of  acquisition  professionals  in  program  management  offices  and 
in  the  decision  chains  might  be  correlated  with  program  success  as  another  area  for 
further  study.  A  study  on  the  correlation  of  training  acquisition  professionals  with 
program  management  certifications,  and  the  success  of  programs  may  be  illustrative  of 
the  knowledge  needed  for  successful  programs. 

The  volume  of  contractual,  legal  and  acquisition  oversight  complicates  approvals 
of  program  documentation  and  has  impacts  on  schedules.121  An  area  of  study  would  be 
the  impacts  of  congressional  intervention  that  micro-manages  the  system  with  oversight 
processes  and  the  impacts  of  those  interventions  on  the  program  success. 
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Acquisition  program  performance  can  and  should  be  improved.  Cost  and 
schedule  overruns  due  to  immature  technology  or  requirements  instability  should  not 
occur  on  a  regular  basis.  And  yet  on  the  FCS  and  the  F-22  approvals  for  entry  into 
EMD  with  both  immature  technologies  and  requirements  changes  occurred.  Risk 
management  can  provide  a  method  to  identify  the  risks  with  immature  technology  and 
requirements  instability.  However,  these  tools  need  to  be  used  with  courageous 
decisions  by  program  managers  and  decision  makers  at  service  and  DOD  leadership 
alike  to  disapprove  programs  that  are  not  ready  for  EMD.  Better  program  management 
will  improve  fiscal  and  schedule  performance.  Other  elements  of  the  acquisition  system 
need  improvement  such  as  encouraging  Congress  to  allow  portfolio  funding  flexibility 
and  the  PPBE  linking  multiyear  funding  commitment  to  EMD  start.  As  Secretary  Gates 
stated  on  release  of  the  2010  DOD  budget  and  in  front  of  congress:  “We  must 
constantly  guard  against  so-called  “requirements  creep,”  validate  the  maturity  of 
technology  at  milestones,  fund  programs  to  independent  cost  estimates,  and  demand 
stricter  contract  terms  and  conditions.  I  am  confident  that  if  we  stick  to  these  steps,  we 
will  significantly  improve  the  performance  of  our  defense  acquisition  programs.  But  it 
takes  more  than  mere  pronouncements  or  fancy  studies  or  reports.  It  takes  acting  on 
these  principles  by  making  tough  decisions  [emphasis  added]  and  sticking  to  them 
going  forward.”122 
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